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(57) Abstract: Apparatus and methods are 

provided that employ one or more of a variety of 
techniques for reducing the time required to display 
high resolution images on a high dynamic range 
display having a light source layer and a display 
layer. In one technique, the image resolution 
is reduced, an effective luminance pattern is 
determined for the reduced resolution image, and 
the resolution of the effective luminance pattern is 
then increased to the resolution of the display layer. 
In another technique, the light source layer's point 
spread function is decomposed into a plurality of 
components, and an effective luminance pattern 
is determined for each component- The effective 
luminance patterns are then combined to produce a 
total effective luminance pattern. Additional image 
display time reduction techniques are provided. 
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H APTD IMAGE RENDERTNQ ON DU AT.-1UOT)IILATOR 

DISPLAYS 

r.ross-Reference to Related Application 
5 [0001] . This application claims priority from United States patent 

application No. 60/591,829 filed on 27 July 2004 and entitled RAPID 

FRAME RENDERING FOR HIGH DYNAMIC RANGE DISPLAYS. 

For purposes of the United States of America, this application claims 
^ the benefit under 35 U.S.C.§119 of United States patent application No. 
10 60/591 ,829 filed on 27 July 2004 and entitled RAPID FRAME 

RENDERING FOR HIGH DYNAMIC RANGE DISPLAYS. 

Technical Field 

[0002] This invention pertains to systems and methods for 
15 displaying images on displays of the type that have two modulators. A 
first modulator produces a light pattern and a second modulator 
modulates the light pattern produced by the first modulator to yield an 
image. 

20 Backgroimd 

[0003] International patent publication WO 02/069030 published 6 
September 2002 and international patent publication WO 03/077013 
published 18 September 2003, both of which are incorporated by 
reference herein, disclose displays which have a modulated light source 

25 layer and a modulated display layer. The modulated light source layer is 
driven to produce a comparatively low-resolution representation of an 
image. The low-resolution representation is modulated by the display 
layer to provide a higher resolution image which can be viewed by an 
observer. The light source layer may comprise a matrix of actively 

30 modulated light sources, such as light emitting diodes (LEDs). The 

display layer, which is positioned and aligned in front of the light source 
layer, may be a liquid crystal display (LCD). 
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[0004] If the two layers have different spatial resolutions (e.g. the 
light source layer's resolution may be about 0.1% that of the display 
layer) then both software correction methods and psychological effects 
(such as veiling luminance) prevent the viewer from noticing the 
5 resolution mismatch. 

[0005] Electronic systems for driving light modulators such as 
arrays of LEDs or LCD panels are well understood to those skilled in 
the art. For example, LCD computer displays and televisions are 

10 commercially available. Such displays and televisions mclude curcuitry 
for controlling the amount of light transmitted by individual pixels m an 
LCD panel. The task of deriving driving from image data signals to 
control a light source layer and display layer can be computationally 
expensive. Deriving such signals can be executed by a processor of a 

15 computer's video/graphics card, or by some other appropriate processor 
integral to a computer, to the display itself or to a secondary device. 

[0006] The task of deriving from image data signals to control a 
light source layer and display layer can be computationally expensive. 

20 Deriving such signals can be executed by a processor of a computer's 
video/graphics card, or by some other appropriate processor integral to 
a computer, to the display itself or to a secondary device. Performance 
lunitations of the processor can undesirably limit the rate at which 
successive image frames can be displayed. For example, if the 

25 processor is not powerful enough to process incoming video data at the 
frame rate of the video data then an observer may detect small pauses 
between successive frames of a video image such as a movie. This can 
distract the observer and negatively affecting the observer's unage 
viewing experience. 



30 
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[0007] There is a need for practical, cost effective and efficient 
systems for displaying images on displays of the general type described 
above. 

5 Brief Description of Drawings 

[0008] The appended drawings illustrate non-limiting embodiments 
of the invention. 

[0009] Figure 1 graphically depicts segmentation of a point spread 
function (PSF) into narrow and wide base Gaussian segments. 
10 [0010] Figures 2A, 2B and 2C graphically depict the splitting of a 
16-bit point spread function (PSF) into two 8-bit (high and low byte) 
segments. 

[0011] Figure 3 graphically depicts the transitional behaviour of 
8-bit high and low byte point spread function values relative to a 16-bit 
15 range. 

[0012] Figure 4 graphically depicts high and low byte point spread 
functions corresponding to the point spread function depicted in Figure 
1. 

[0013] Figure 5 graphically depicts application of an iteratively- 
20 derived interpolation function to derive an interpolated effective 
luminance pattern (ELP) closely approximating an actual effective 
luminance pattern (ELP). 

[0014] Figure 6 is a schematic diagram of a display. 
[0015] Figure 7 is a flowchart illustrating a method for displaying 
25 an image on a display having a controllable light source layer and a 
controllable display layer. 

[0016] Figure 8 is a flowchart illustratmg a method for 
determining an effective luminance pattern. 
[0017] Figure 9 is a flowchart illustrating a method for 
30 determining an effective luminance pattern or a component of an 
effective luminance pattern. 
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Description 

[0018] Throughout the following description, specific details are 
set forth in order to provide a more thorough understanding of the 
invention. However, the invention may be practiced without these 
5 particulars. In other instances, well known elements have not been 
shown or described in detail to avoid unnecessarily obscuring the 
invention. Accordingly, the specification and drawings are to be 
regarded in an illustrative, rather than a restrictive, sense. 

10 [0019] The invention may be applied in a wide range of 

applications wherein an image is displayed by producing a light pattern 
that is determined at least in part by image data, and modulating the 
light pattern to yield an image. The light pattern may be produced by 
any suitable apparatus. Some examples include: 
15 • A plurality of light sources driven by driver circuits that permit 
brightnesses of the light sources to be varied. 
• A fixed or variable light source combined with a reflection type 
or transmission type modulator that modulates light from the light 
source. 

20 The following description relates to non-limiting example embodiments 
in which the light pattern is produced on one side of an LCD panel by 
an array of light-emitting diodes and the LCD panel is controlled to 
modulate the light of the light pattern to produce a viewable image. In 
this example, the array of LEDs can be considered to constitute a first 

25 modulator and the LCD panel constitutes a second modulator. 

[0020] In general, rendering image frames or a frame set for 
display on an LED/LCD layer display entails the following 
computational steps: 
30 1 . Obtaining image data (which may be full screen or partial screen 

image data). 
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2. Deriving from the image data appropriate driving values for each 
LED of the first modulator, using suitable techniques well known 
to persons skilled in the art (e.g. nearest neighbour interpolation 
which may be based on factors such as intensity and colour). 
5 3. The derived LED driving values and the point spread functions of 
LEDs on the LED layer as well as the characteristics of any 
layers between the LED layer and the LCD layer are used to 
determme the effective luminance pattern which will result on the 
LCD layer when the LED drivmg values are applied to the LED 

10 layer. 

4. The image defined by the image data is then divided by the 
effective luminance pattern to obtain raw modulation data for the 
LCD layer. 

5. In some cases, the raw modulation data is modified to address 

15 issues such as non-lmearities or other artifacts arising in either of 

the LED or LCD layers. These issues can be deah with using 
suitable techniques well known to persons skilled m the art (e.g. 
scaling, gamma correction, value replacement operations, etc.). 
For example, creating the modified modulation data may mvolve 

20 altering the raw modulation data to match a gamma correction 

curve or other specific characteristics of the LCD layer. 

6. Final modulation data for the LCD (which may be the raw 
modulation data or the modified modulation data) and the driving 
data for the LEDs are applied to drive the LCD and LED layers 

25 to produce the desired image. 

[0021] Various ways to reduce the computational cost of (i.e. to 
speed up) generating the final modulation data for use in displaying 
images are described herein. These include: 
30 • Performing at least some parts of the computation in a lower 

precision domain (for example, by performing computations in 
the 8-bit domam instead of in the 16-bit domam); and. 
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• Implementing one or more of the options for efficiently 

establishing an effective luminance pattern that are described 
herein. 

While these techniques may be implemented individually, any suitable 
5 combinations of the techniques described herem may be used. 

Hffective Luminance Pattern Determination 

[0022] The point spread function of each LED in an LED layer is 
determmed by the geometry of the LED. A srniple technique for 

10 determining an LED layer's total effective lummance pattern is to 
initially multiply each LED's pomt spread function (specifically, the 
point spread function of the Ught which is emitted by the LED and 
passes through all optical structures between the LED and LCD layers) 
by a selected LED driving value and by an appropriate scaling 

15 parameter to obtain the LED ' s effective luminance contribution, for that 
drivmg value, to each pixel on the LCD layer. 

[0023] In this way, the luminance contributions of every LED in 
the LED layer can be determined and summed to obtam the total 

20 effective luminance pattern, on the LCD layer, that will be produced 
when the selected driving values are applied to the LED layer. 
However, these multiplication and addition operations are very 
computationally expensive (i.e. time consuming), because the effective 
lummance pattern must be determined to the same spatial resolution as 

25 the LCD layer in order to facilitate the division operation of step 4 
above. 

[0024] The computational expense is especially great if the LED 
point spread function has a very wide "support." The "support" of an 
30 LED point spread function is the number of LCD pixels that are 

illuminated in a non-negligible amount by an LED. The support can be 
specified m terms of a radius, measured in LCD layer pixels, at which 
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the LED point spread function becomes so small that is perceptually 
irrelevant to an observer. The support corresponds to a number of LCD 
pixels that are illuminated in a significant amount by each LED. 

5 [0025] For example, consider a hexagonal LED array in which the 
centre of each LED is spaced from the immediately adjacent LEDs by a 
distance equal to 50 of the LCD layer's pixels. If each LED has a point 
spread function having a support of 150 LCD pixels then each pixel in 
the center portion of the LCD layer will be illuminated by light from 

10 approximately 35 of the LEDs. Calculation of the effective luminance 
pattern for this example accordingly requires 35 operations for each 
pixel of the LCD layer, in order to account for the light contributed to 
each pixel by each relevant LED. Where the LCD layer has a high 
spatial resolution, this is very computationally expensive (i.e. time 

15 consuming). 

Pftsnhitinn Reduction 

[00261 The time required to determine the effective luminance 
pattern produced on the LCD can be reduced by computmg the effective 

20 luminance pattern at a reduced spatial resolution that is lower than that 
of the high resolution image which is to appear on the LCD layer. This 
is feasible because the point spread functions of individual light sources 
are generally smoothly varying. Therefore, the effective luminance 
pattern will be relatively slowly varying at the resolution of the LCD. It 

25 is accordingly possible to compute the effective luminance pattern at a 
lower resolution and then to scale the effective luminance pattern up to a 
desired higher resolution, without introducing significant artifacts. 

[0027] The scaling may be done using suitable linear, Gaussian or 
30 other interpolation techniques. Such spatial resolution reduction yields 
an approximately linear decrease in the computational cost of 
establishing the effective luminance pattern. Many available 
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interpolation methods that can be used to scale up an effective 
luminance pattern computed at a lower resolution are computationally 
inexpensive as compared to the computational cost of computing the 
effective luminance pattern at the resolution of the LCD or other second 
5 light modulator. 

[0028] Using the foregomg example, a 10-times resolution 
reduction in both the width and height directions yields an approximate 
100-times reduction in computational cost. This is because the total 

10 number of pixels in the reduced resolution image is 100-times fewer 
than the total number of pixels in the high resolution image which is to 
appear on the LCD layer. Each pixel in the reduced resolution image 
still receives light from 35 LEDs, necessitating 35 computational 
operations per pixel— but those operations are applied to 100-times 

15 fewer pixels in comparison to a case in which the computations are 

performed separately for every pixel in the actual high resolution image 
which is to appear on the LCD layer. 

Point S pread Function Decomposition 

20 [0029] The computational cost of image rendering can also be 
reduced by decomposing the point spread function of each light source 
(e.g. each LED) into several components (e.g. by performing a 
Gaussian decomposition) in such a way that the recombination of all of 
the components yields the original pomt spread function. An effective 

25 luminance pattern can then be determined separately for each 

component. Once an effective luminance pattern has been determined 
for each component, those effective luminance patterns can be combined 
to produce a total effective luminance pattern. The combination may be 
made by summing, for example. 



30 
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[0030] Computing the effective luminance patterns contributed by 
the components may be performed at the resolution of the LCD layer or 
at a reduced resolution, as described above. 

5 [0031] A speed benefit is attained even if the effective luminance 
pattern for each component is computed at the resolution of the LCD 
layer since hardware components specially adapted to perform rapid 
computations based upon standard point spread functions (e.g. Gaussian 
point spread functions) are commercially available. Such hardware 

10 components are not normally commercially available for the typically 
non-standard point spread function of the actual LEDs in the display's 
LED layer— necessitating resort to considerably slower computational 
techniques using general purpose processors. 

15 [0032] A greater speed benefit is attained if the resolution 

reduction technique described above is used to determine an effective 
lummance pattern for each component. Moreover, different spatial 
resolutions can be applied to different components of the point spread 
functions to yield even greater speed benefits. For example. Figure 1 

20 depicts (solid line) an example LED point spread function havmg a steep 
central portion 10 and a wide tail portion 12. In this situation, the 
actual point spread function can be decomposed into a narrow base 
Gaussian component 14A and a wide base Gaussian component 14B, as 
depicted. 

25 

[0033] The wide base Gaussian component 14B (dotted line) 
contributes relatively little image intensity, m comparison to narrow 
base Gaussian segment 14A (dashed Ime). Further, wide base Gaussian 
component 14B is more slowly varying than narrow base Gaussian 
30 component 14A. Accordingly, an effective luminance pattern for narrow 
base Gaussian component 14A may be determined at a moderately high 
spatial resolution while an effective luminance pattern for the wide base 
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Gaussian component 14B can be computed at a significantly lower 
spatial resolution. This preserves a substantial portion of the image 
intensity information contained in narrow base Gaussian component 
14A and is still relatively fast since the effective support of the narrow 

5 base Gaussian segment is small and thus few LCD pixels are covered by 
that component. By contrast, since wide base Gaussian component 14B 
contains relatively little image intensity information, that component can 
be processed relatively quickly at low resolution without substantially 
degrading the resolution of the total effective luminance pattern 

10 produced by combining the patterns derived for each component. 

R-hit Segmentation 

[0034] Image data is typically provided in 16-bit word form. 
High-end (i.e. more expensive) graphic processors typically perform 

15 computations in the 16-bit domain. Such processors may have dedicated 
16-bit or floating point arithmetic units that can perform 16-bit 
operations quickly. The need for a high-end processor capable of 
performing 16-bit operations quickly can be alleviated by computing the 
effective luminance pattern in the 8-bit domain. Such computations can 

20 be performed reasonably quickly by less expensive processors. 

[0035] Each LED's point spread function is a two dimensional 
function of intensity versus distance relative to the center of the LED. 
Such a point spread function may be characterized by a plurality of 
25 16-bit data words. Where the point spread function is represented by a 
look up table, many 16-bit values are required to define the point spread 
function; for example, one value may be provided for every LCD pixel 
lying on or within a circle centered on the LED and having a radius 
corresponding to the support of the point spread function. 

30 

[0036] Each one of those 16-bit data words has an 8-bit high byte 
component and an 8-bit low byte component (any 16-bit value A can be 
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divided into two 8-bit values B and C such that>4 = 5 + C, where 
B is the "high byte" and C is the "low byte"). The 8-bit values are 
preferably extracted only after all necessary scaling and manipulation 
operations have been applied to the input 16-bit data. Figure 2A depicts 
5 a 16-bit point spread function; Figures 2B and 2C respectively depict 
the 8-bit high and low byte components of the Figure 2A 16-bit point 
spread function. 

[0037] A 16-bit data word is capable of representing integer values 

10 from 2''-l to 2^^-l (i.e. from 0 to 65535). An 8-bit byte is capable of 
representmg mteger values from 2°-l to 2^-1 (i.e. from 0 to 255). The 
"support" (as previously defined) of a point spread function 
characterized by an 8-bit high byte component is much smaller 
(narrower) than the support of the point spread function as a whole. 

15 This is because the 8-bit high byte component reaches the lowest value 
(zero) of its 255 possible values, when the 16-bit data word 
characterizing the point spread function as a whole reaches the value 
255 out of its range of 65535 possible values. The remaining 255 
values are provided by the low byte component with the high byte 

20 component's value equal to zero. The effective luminance pattern 
corresponding to the narrow base 8-bit high byte component can 
accordingly be rapidly determined, without substantial loss of image 
intensity information. The resolution reduction and/or other techniques 
described above may be used to further speed up the determination of 

25 the effective luminance pattern for the 8-bit high byte component. 

[0038] The support of a point spread function characterized by an 
8-bit low byte component is comparatively wide. Specifically, although 
the 8-bit low byte component has only 255 possible values, those values 
30 decrease from 255 to 0 (out of 65535 values for the point spread 

function as a whole) and those 255 values correspond to the 255 lowest 
intensity levels (i.e. levels at which the value of the high byte 
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component is equal to zero). Those 255 levels represent the valued of 
the point spread function in its peripheral parts. 

[0039] The low byte component can be separated into two regions. 

5 A central region, lying within the boundary on which the point spread 
function characterized by the high byte component reaches zero. In the 
central region the low-byte component typically varies in an irregular 
saw-tooth pattern (as depicted in Figure 3) if the original 16-bit point 
spread function is reasonably smooth. This is because, m the central 

10 region, the portion of the point spread function characterized by the low 
byte component augments the portion of the point spread function 
characterized by the high byte component. 

[0040] For example, consider a transition from the 16-bit value 
15 10239 to the 16-bit value 9728. The 16-bit value 10239 has a high byte 
component value of 39 and a low byte component value of 255 (i.e. 
39*256-1-255 = 10239). Consequently, the low byte component's 
contribution to the point spread function is initially 255 and the high 
byte component's contribution is mitially 39. The value of the high byte 
20 component's contribution remains at 39, while the value of the low byte 
component's contribution smoothly decreases from 255 to 0— the point 
at which the original 16-bit point spread function has the value 9984 
(i.e. 39*256+0). The value of the high byte component's contribution 
to the point spread function then changes smoothly from 39 to 38, but 
25 that change is accompanied by an abrupt change (from 0 to 255) in the 
value of the low byte component's contribution to the point spread 
function. 

[0041] As seen in Figure 4, inside a radius R of the original point 
30 spread function (and where the value of the high byte component's 
contribution to the point spread function is non-zero) the resulting 
saw-tooth pattern of the low byte component's contribution to the point 
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Spread function is characteristic of the original point spread function. 
Outside the radius R, the value of the high byte component's 
contribution to the point spread function is zero, and the value of die 
low byte component's contribution changes smoothly. 

5 

[0042] The contributions from the low-byte component of the point 
spread function can be processed differentiy m these two regions (i.e. 
the regions inside and outside the radius R) to avoid unwanted artifacts. 
For example, to preserve a substantial portion of the image intensity 

10 information contained in the region inside the radius R, the effective 
lummance pattern for that region is preferably determined using the 
same relatively high resolution used to determme the effective 
luminance pattern for the high byte component's contribution to the 
point spread function, as previously described. By contrast, the 

15 effective lummance pattern for the region outside the radius R can be 
determined using a much lower resolution, witiiout substantial loss of 
image intensity information. 

[0043] After the three point spread function segments (i.e. the high 
20 byte component, the region of the low byte component inside the radius 
R, and the region of the low byte component outside the radius R) have 
been processed as aforesaid, the results are individually up-sampled to 
match the resolution of the LCD layer, then recombined with 
appropriate scalmg factors being applied. Recombination typically 
25 involves summation of the values for the two low byte component 
regions and the value for the high byte component, after the value for 
the high byte component has been multiplied by 256. 

Inter polation 

30 [0044] If an effective luminance pattern value is determined usmg 
a resolution lower than the resolution of the LCD layer, it is necessary 
to up-sample that value to match the resolution of the LCD layer. 
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Interpolation techniques for up-sampling low resolution images into high 
resolution images are well known, with both linear and Gaussian based 
techniques being common. Although such prior art techniques can be 
used in conjunction with the above described techniques, accuracy, or 

5 speed, or both may be improved by utilizing an interpolation technique 
which is optimized for a particular display configuration. Optimization 
facilities higher resolution unage compression, minimizes introduction 
of unwanted interpolation artifacts, and reduces the image rendering 
time. In extreme cases, an interpolation technique can be used to 

10 reduce the resolution of the effective luminance pattern resolution to 
match the resolution of the LED layer. 

[0045] Prior art interpolation techniques are often restricted to use 
with specific pre-interpolation data, or to use with specific interpolation 

15 functions. The interpolation techniques used to match the resolution of 
the effective luminance pattern to that of the LCD display do not need to 
satisfy such restrictions, provided convolution of the pre-interpolation 
data with the selected interpolation function will yield an effective 
luminance pattern having adequate similarity to the actual effective 

20 luminance pattern. 

[0046] The required degree of similarity depends on the display 
application. Different appHcations require different degrees of 
similarity— in some applications relatively small deviations may 

25 unacceptably distract an observer, whereas larger deviations may be 
tolerable in other applications (such as applications involving television 
or computer game images in which relatively large deviations 
nonetheless yield images of quality acceptable to most observers). 
Consequently, it is not necessary to apply the interpolation technique 

30 directly to the actual LED driving values or to the actual LED point 
spread function. 
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[0047] For example, Figure 5 depicts the result obtained by using 
an iteratively-derived interpolation technique to reduce the resolution of 
the effective luminance pattern to match the resolution of the LED 
layer. The pixel values at the LED layer's resolution are not the LED 

5 driving values— they are the luminance values of the effective luminance 
pattern before interpolation. The interpolation function can be 
determined using standard iteration methods and a random starting 
condition. As seen in Figure 5, convolution of the iteratively-derived 
interpolation function with the effective luminance pattern values yields 

10 results which are reasonably close to the actual effective luminance 
pattern. 

[0048] Many different interpolation techniques can be used. There 
need not be any correlation between the interpolation function and the 
15 LEDs' point spread function, the LED driving values, or any other 

characteristic of the display, provided the selected interpolation function 
and the input parameters selected for use with that function yields a 
result reasonably close to the actual effective luminance pattern. 

20 Exam ple Embodiments 

[0049] Figures 6 to show some example embodiments of the 
invention. Figure 6 shows a display 30 comprising a modulated light 
source layer 32 and a display layer 34. Light source layer 32 may 
comprise, for example: 

25 • an array of controllable light sources such as LEDs; 

• a fixed-intensity light source and a light modulator disposed to 
spatially modulate the intensity of light from the light source; 

• some combination of these. 

In the illustrated embodiment, light source layer 32 comprises an array 
30 of LEDs 33. 
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[0050] Display layer 34 comprises a light modulator that further 
spatially modulates the intensity of light incident on display layer 34 
from light source layer 32.Display layer 34 may comprise an LCD 
panel or other transmission-type light modulator, for example. Display 

5 layer 34 typically has a resolution higher than a resolution of light 
source layer 32. Optical structures 36 suitable for carrying light from 
light source layer 32 to display layer 34 may be provided between light 
source layer 32 and display layer 34. Optical structures 36 may 
comprise elements such as open space, light diffusers, colUmators, and 

10 the like. 

[0051] In the illustrated embodiment a controller 40 comprising a 
data processor 42 and suitable interface electronics 44A for controUmg 
light source layer 32 and 44B for controlling display layer 34 receives 

15 unage data 46 specifying images to be displayed on display 30. 

Controller 40 drives the light emitters (e.g. LEDs 33) of light source 
layer 34 and the pixels 35 of display layer 34 to produce the desned 
image for viewing by a person or persons. A program store 46 
accessible to processor 42 contams software instructions that, when 

20 executed by processor 42 cause processor 42 to execute a method as 
described herein. 

[0052] Controller 40 may comprise a suitably programmed 
computer having appropriate software/hardware interfaces for 
25 controlUng light source layer 32 and display layer 34 to display an 
image specified by image data 48. 

[0053] Figure 7 shows a method 50 for displaying unage data on a 
display of the general type shown in Figure 6. Method 50 begins by 
30 receiving image data 48 at block 52. In block 54 first driving signals for 
light source layer 32 are derived from image data 48. Suitable known 
methods may be applied to obtain the first driving signals in block 54. 
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[0054] In block 56 method 50 computes an effective luminance 
pattern. The effective luminance pattern may be computed from the first 
driving signals and known point spread functions for the light sources 

5 of light source layer 32. Block 56 computes the effective luminance 
pattern at a resolution that is lower than a resolution of display layer 34. 
For example, block 56 may compute the effective luminance pattern at a 
resolution that is a factor of 4 or more smaller in each dimension (in 
some embodiments a factor in the range of 4 to 16 smaller m each 

10 dimension) than the resolution of display layer 34. 

[0055] In block 60 the effective luminance pattern computed in 
block 58 is upsampled to the resolution of display layer 34. This may be 
done through the use of any suitable interpolation technique for 
15 example. In block 62 second driving signals for the display layer are 
determined from the upsampled effective luminance pattern and the 
image data. The second driving signals may also take into account 
known characteristics of the display layer and any desired image 
corrections, colour corrections or the like. 

20 

[0056] In block 64 the first driving signal obtained in block 54 is 
applied to the light source layer and the second driving signals of block 
62 are applied to the display layer to display an unage for viewing. 

25 [0057] Figure 8 shows a method 70 for computing an effective 
luminance pattern. Method 70 may be applied within block 56 of 
method 50 or may be used in other contexts. Method 70 begins by 
computing an ELP for each component of the point spread function for 
the light sources of light source layer 32 (blocks 72A, 72B and 72C - 

30 collectively blocks 72) . Blocks 72 may be performed in any sequence or 
may be performed in parallel with one another. Figure 8 shows three 
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PSF components 73A, 73B and 73C and three corresponding blocks 72. 
The method could be practised with two or more PSF components 73. 

[0058] The components of the point spread function (PSF) will 
5 typically have been predetermmed. A representation of each component 
is stored in a location accessible to processor 42. Each of blocks 72 may 
comprise, for each light source of light source layer 32, multiplying 
values that define a component of the point spread function by a value 
representmg the intensity of the light source. In block 74 the effective 
10 luminance patters determined m blocks 72 are combined, for example 
by summing, to yield an overall estimate of the effective luminance 
pattern that would be produced by applying the furst driving signals to 
light source layer 32. 

15 [0059] Figure 9 illustrates a method 80 that may be applied for 
computing effective luminance patterns. Method 80 may be applied to: 

• computing the effective luminance pattern in block 56 of method 
50; or 

• computing the effective luminance patterns for individual 

20 components of a point spread function m blocks 72 of method 70; 

or 

• applied in other contexts. 

[0060] Method 80 begins in block 82 with data characterizing a 
25 point spread function (or a PSF component) for a light source of light 
source layer 32 and data indicative of how intensely the light source will 
operate under the control of the first driving signals. Method 80 
combines these values (e.g. by multiplying them together) to obtam a set 
of values characterizing the contribution of the light source to the 
30 effective luminance pattern at various spatial locations. 
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[0061] Block 84 obtains high-order and low-order components of 
the resulting values. In some embodiments, the resulting values are 16- 
bit words, the high-order component is an 8-bit byte and the low-order 
component is an 8-bit byte. 

5 

[0062] Contributions to the ELP are determined separately for the 
high-order and low-order components m blocks 86 and 88. For each 
light source, the area of support for which values are included in the 
high-order contribution of 86 is typically significantly smaller than the 
10 area of support for which values are included in the low-order 
contribution of block 88. 

[0063] Block 88 typically computes the low-order contribution for 
points located within the area of support of the high-order contribution 
15 (block 90) separately than for points located outside of the area of 

support of the high-order contribution (block 92). Blocks 86, 90 and 92 
may be performed in any order or simultaneously. 

[0064] In block 94 the contributions from blocks 86, 90 and 92 are 
20 combined to yield an overall ELP. The computations in blocks 86 90 
and 92 may be performed primarily or entirely in the 8-bit domain (i.e. 
using 8-bit operations on 8-bit operands) in the case that the high-order 
and low-order components are 8-bit bytes or smaller. 

25 [0065] Certam implementations of the invention comprise 

computer processors which execute software instructions which cause 
the processors to perform a method of the invention. For example, one 
or more processors m a computer or other display controller may 
implement the methods of Figures 7, 8 or 9 by executing software 

30 instructions in a program memory accessible to the processors. The 
invention may also be provided in the form of a program product. The 
program product may comprise any medium which carries a set of 
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computer-readable signals comprising instructions which, when 
executed by a data processor, cause the data processor to execute a 
method of the invention. Program products according to the mvention 
may be in any of a wide variety of forms. The program product may 

5 comprise, for example, physical media such as magnetic data storage 
media including floppy diskettes, hard disk drives, optical data storage 
media mcluding CD ROMs, DVDs, electronic data storage media 
mcluding ROMs, flash RAM, or the like or transmission-type media 
such as digital or analog communication links. The computer-readable 

10 signals on the program product may optionally be compressed or 
encrypted. 

[0066] Where a component (e.g. a member, part, assembly, device, 
processor, controller, collimator, circuit, etc.) is referred to above, unless 

15 otherwise indicated, reference to that component (including a reference 
to a "means") should be interpreted as including as equivalents of that 
component any component which performs the function of the described 
component (i.e., that is functionally equivalent), including components 
which are not structurally equivalent to the disclosed structure which 

20 performs the function in the illustrated exemplary embodiments of the 
invention. 

[0067] As will be apparent to those skilled in the art in the light of 
the foregoing disclosure, many alterations and modifications are 
25 possible m the practice of this invention without departmg from the 
spirit or scope thereof. For example, 

• The light source layer may comprise a number of different types 
of Ught source that have point spread functions different from one 
another; 

30 • The display may comprise a colour display and the computations 
described above may be performed separately for each of a 
number of colours. 
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[0068] While a number of example aspects and embodiments have 
been discussed above, those of skill in the art will recognize certain 
modifications, permutations, additions and sub-combinations thereof. It 
5 is therefore intended that the following appended claims and claims 
hereafter introduced are interpreted to include all such modifications, 
permutations, additions and sub-combmations as are within their true 
scope. 
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What is claimed is: 

1. A method for displaying an image on a display comprising a light 
source layer and a display layer, the method comprising: 

5 determining driving values for light sources of the light 

source layer; 

determining an effective luminance pattern of the light 
source layer at a first spatial resolution lower than a spatial 
resolution of the display layer; and, 
10 increasmg the spatial resolution of the effective luminance 

pattern to a second spatial resolution corresponding to the 
resolution of the display layer. 

2. A method according to claim 1 wherein the resolution of the 

15 display layer is at least 4 times greater than the resolution used in 

determining the effective luminance pattern m at least one 
dimension. 

3 . A method according to claim 2 wherein the resolution of the 

20 display layer is at least 8 times greater than the resolution used in 

determining the effective Immnance pattern m each of two 
dimensions. 

4. A method according to claim 1, 2 or 3 wherein increasing the 
25 spatial resolution of the effective luminance pattern comprises 

performing interpolation on data defining the effective lummance 
pattern. 

5. A method according to any one of claims 1 to 4 wherein 

30 determining the effective lummance pattern of the light source 

layer comprises: 
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determining a contribution to the effective luminance 
pattern for each of a plurality of components of a point spread 
function for light sources of the light source layer; and, 

combining the contributions to the effective luminance 
5 pattern of each of the components. 

6. A method according to claim 5 wherein each of the components is 
a Gaussian component. 

10 7. A method according to claim 5 or claim 6 wherein the pomt 
spread function is the sum of all of the components of the 
plurality of components. 

8. A method according to one of claims 5 to 7 wherein each of the 
15 components is represented at the first spatial resolution. 

9. A method according to one of claims 5 to 7 wherein two or more 
of the components are represented at spatial resolutions different 
from one another. 

20 

10. A method according to claim 8 or 9 comprising, before 
combining the contributions to the effective luminance pattern, 
increasing the spatial resolution of each of the components to the 
second spatial resolution. 

25 

11. A method according to clarni 5 or 6 wherein combining the 
contributions to the effective luminance pattern comprises 
applying a mathematical inverse of an operation applied to 
decompose the point spread function into the plurality of 

30 components. 
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12. A method accofding to any one of claims 5 to 11 wherein 
determining a contribution to the effective luminance pattern for 
each of a plurality of components of a point spread function is 
performed over a different support area for each of two of the 

5 components of the point spread function. 

13. A method according to any one of claims 1 to 12 wherein 
determining the effective luminance pattern of the light source 
layer comprises: 

10 for each of a plurality of light sources of the light source 

layer: 

separately determining contributions to the effective 
luminance pattern of higher-order and lower-order parts of a set 
of point spread function values; and, 
15 combining the contributions to the effective luminance 

pattern of the higher-order and lower-order point spread function 
values. 

14. A method according to claim 13 whereui the point spread function 
20 values comprise 16 bit words and the higher-order and lower- 
order parts of the set of point spread function values comprise 8- 
bit words. 

15. A method according to claim 13 or claim 14 wherein determining 
25 contributions to the effective luminance pattern of higher-order 

and lower-order parts of the set of point spread function values is 
performed over a larger support area for the lower-order parts of 
the set of point spread function values than for the higher-order 
parts of the set of point spread function values. 



30 



16. A method according to claim 15 wherein determining 

contributions to the effective luminance pattern of the lower-order 
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parts of the set of point spread function values comprises 
separately determining a contribution for each of: 

the intersection of the support area of the higher-order and 
lower-order parts of the point spread function values; and, 
5 the part of the support area for the lower-order parts of the 

point spread function values that is outside of the support area for 
the higher-order parts of the point spread function values. 

17. A method according to one of claims 13 to 16 comprising 
10 identifying a support area for the higher-order parts of the point 

spread function values by determining a radius R at which the 
point spread function of the higher-order part of the point spread 
function values is equal to zero. 

15 18. A method according to any of claims 13 to 18 comprising 

determining the contribution to the effective luminance pattern of 
the lower-order parts of the set of point spread function values at 
different resolutions within and outside of the support area of the 
higher-order parts of the point spread function values. 

20 

19. A method according to claim 18 comprising determining the 

contribution to the effective luminance pattern of the lower-order 
parts of the set of point spread function values at a higher 
resolution within the support area of the higher-order parts of the 
25 point spread function values and at a lower resolution outside of 

the support area of the higher-order parts of the point spread 
function values . 



20. 

30 



A computer-readable medixrai carrying computer instructions that, 
when executed by a processor, cause the processor to execute a 
method according to any one of claims 1 to 19. 
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21 . Apparatus for controlling a display comprising a light source 
layer and a display layer, the apparatus comprising: 
a controller configured to: 

determine first driving values for light sources of the 
5 light source layer from image data; 

determine an effective luminance pattern of the Ught 
source layer at a first spatial resolution lower than a spatial 
resolution of the display layer; 

increase the spatial resolution of the effective 
10 luminance pattern to a second spatial resolution 

correspondmg to the resolution of the display layer; and, 

determine second drivuig values for the display layer 
based on at least the unage data and the effective lummance 
pattern; 

15 a first interface connectable to the light source layer to 

apply the first driving values to the light source layer; and, 

a second interface connectable to the display layer to apply 
the second driving values to the display layer. 

20 22. Apparatus according to claim 21 comprising a light source layer 
connected to the first mterface and a display layer connected to 
the second interface. 

23 . Apparatus according to claim 22 wherein the light source layer 
25 comprises a plurality of individually-controllable light sources. 

24. Apparatus according to claim 22 wherein the light source layer 
comprises an array of light-emitting diodes. 



30 25. 



Apparatus according to claun 22 wherein the light source layer 
comprises a light source and a modulator disposed to modulate 
light emitted by the light source. 
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26. Apparatus according to any one of claims 21 to 25 wherein the 
display layer comprises a transmission-type modulator havmg a 
plurality of individually-controllable pixels. 

5 27. Apparatus according to any one of claims 21 to 26 wherein the 
display layer comprises an LCD panel. 

28. Apparatus according to any one of clauns 21 to 25 wherein a 
resolution of the display layer is at least 4 times greater than the 

10 first spatial resolution. 

29. Apparatus according to claim 28 wherein the resolution of the 
display layer is at least 8 times greater than the furst spatial 
resolution in each of two dimensions. 

15 

30. Apparatus according to any one of clauns 21 to 29 comprising a 
means for increasing the spatial resolution of the effective 
luminance pattern by performing interpolation on data defining 
the effective luminance pattern. 

20 

31. Apparatus according to any one of claims 21 to 31 comprising a 
data store accessible to the controller and containing information 
defining a plurality of components of a point spread function for 
light sources of the light source layer wherein the controller is 

25 configured to separately evaluate and combine contributions to the 

effective luminance pattern corresponding to each of the 
components. 



32. Apparatus accorduig to claim 31 wherein each of the components 
30 is a Gaussian component. 
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33 . Apparatus according to claim 32 comprising a hardware processor 
providing a function that operates directly on Gaussian 
components. 

5 34. Apparatus according to any one of claims 1 to 33 comprising an 
upsampler for increasing the spatial resolution of the contributions 
to the effective luminance pattern corresponding to each of the 
components to the second spatial resolution. 

10 35. Apparatus according to any one of claims 21 to 34 comprising a 
means for determining a component of the effective luminance 
pattern corresponding to higher-order parts of data and a 
component of the effective luminance pattern corresponding to 
lower-order parts of the data. 

15 

36. Apparatus according to claim 35 wherein the means for 
determining a component of the effective luminance pattern . 
corresponding to higher-order parts of data comprises software 
instructions that cause a processor of the controller to perform 

20 operations primarily in the 8-bit domain. 

37. Apparatus according to claim 35 or 36 wherein the means for 
determining a component of the effective luminance pattern 
corresponding to lower-order parts of data comprises software 

25 instructions that cause a processor of the controller to perform 

operations primarily in the 8-bit domain. 

38. A method for displaying an image on a display comprising a light 
source layer and a display layer, the method comprismg: 

30 determining based at least in part on image data driving 

values for light sources of the light source layer; 
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determining an effective luminance pattern of the light 
source layer by a method including: 

determining a contribution to the effective limiinance 
pattern for each of a plurality of components of a point 
spread function for light sources of the light source layer; 
and, 

combining the contributions to the effective 
luminance pattern of each of the components to yield 
effective luminance pattern data; and, 
determining driving values for the display layer based at 

least in part on the effective luminance pattern data and the image 

data. 

Display apparatus comprising: 
a light source layer 
a display layer, 
a controller configured to: 

determine, based at least m part on image data, first 
driving values for light sources of the light source layer; 

determine an effective luminance pattern of the light 
source layer by a method including: 

determining a contribution to the effective 
luminance pattern for each of a plurality of 
components of a point spread function for light 
sources of the light source layer; and, 

combining the contributions to the effective 
luminance pattern of each of the components to yield 
effective luminance pattern data; and, 
determine second driving values for the display layer based 
at least in part on the effective luminance pattern data and the 
image data; 
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a first interface connected to the light source layer to apply 
the first driving values to the light source layer; and, 

a second interface connected to the display layer to apply 
the second driving values to the display layer. 

5 

40. A method for displaying an image on a display comprising a light 
source layer and a display layer, the method comprising: 

determining based at least in part on image data driving 
values for light sources of the light source layer; 
10 determining an effective luminance pattern of the light 

source layer by a method including: 

for each of a plurality of light sources of the light source 
layer: 

separately determining contributions to the effective 
15 limiinance pattern of higher-order and lower-order parts of 

a set of point spread function values; and, 

combining the contributions to the effective 
luminance pattern of the higher-order and lower-order point 
spread function values to yield effective luminance pattern 

20 data; and, 

determining driving values for the display layer based at 
least in part on the effective luminance pattern data and the image 
data. 



25 41. Display apparatus comprismg: 

a light source layer 
a display layer, 
a controller configured to: 

determine, based at least in part on image data, first 
30 driving values for light sources of the light source layer; 

determine an effective luminance pattern of the light 
source layer by a method including: 
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for each of a plurality of light sources of the 
light source layer: 

separately determining contributions to the effective 
luminance pattern of higher-order and lower-order parts of 
5 a set of point spread fimction values; and, 

combining the contributions to the effective 
luminance pattern of the higher-order and lower-order pomt 
spread function values to yield effective luminance pattern 
data; and, 

10 determine second driving values for the display layer 

based at least in part on the effective luminance pattern data 
and the image data; 

a first interface connected to the light source layer to apply 
the first driving values to the light source layer; and, 
15 a second interface connected to the display layer to apply 

the second driving values to the display layer. 

42. A method comprising any new, useful and inventive step, act, 
combination of steps and/or acts or sub-combmation of steps 
20 and/or acts described herein. 



43. 



Apparatus comprising any new, useful and inventive feature, 
combination of features and/or means, or sub-combination of 
features and/or means disclosed herein. 
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